Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid many failing tests in doctesting #27262

Merged
merged 4 commits into from
Nov 3, 2023
Merged

Avoid many failing tests in doctesting #27262

merged 4 commits into from
Nov 3, 2023

Conversation

ydshieh
Copy link
Collaborator

@ydshieh ydshieh commented Nov 3, 2023

What does this PR do?

Avoid many failing tests in doctesting: currently, src/transformers/models/blip_2/modeling_blip_2.py and docs/source/en/tasks/idefics.md cause problem to other tests

With this PR: the failing list is much shorter

transformers.generation.configuration_utils.GenerationConfig.from_pretrained
transformers.generation.logits_process.ExponentialDecayLengthPenalty
transformers.generation.logits_process.WhisperTimeStampLogitsProcessor
transformers.models.wav2vec2.tokenization_wav2vec2.Wav2Vec2CTCTokenizer.decode
transformers.models.wav2vec2_with_lm.processing_wav2vec2_with_lm.Wav2Vec2ProcessorWithLM.decode
transformers.models.whisper.modeling_whisper.WhisperForCausalLM.forward

Currently failing tests without this PR

transformers.generation.configuration_utils.GenerationConfig.from_pretrained
transformers.generation.logits_process.ExponentialDecayLengthPenalty
transformers.generation.logits_process.WhisperTimeStampLogitsProcessor
transformers.models.mbart.modeling_tf_mbart.TFMBartForConditionalGeneration.call
transformers.models.mbart.modeling_tf_mbart.TFMBartModel.call
transformers.models.mobilevit.modeling_tf_mobilevit.TFMobileViTForSemanticSegmentation.call
transformers.models.opt.modeling_tf_opt.TFOPTForCausalLM.call
transformers.models.opt.modeling_tf_opt.TFOPTModel.call
transformers.models.regnet.modeling_tf_regnet.TFRegNetForImageClassification.call
transformers.models.regnet.modeling_tf_regnet.TFRegNetModel.call
transformers.models.resnet.modeling_tf_resnet.TFResNetForImageClassification.call
transformers.models.resnet.modeling_tf_resnet.TFResNetModel.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForCausalLM.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForMaskedLM.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForMultipleChoice.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForQuestionAnswering.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForSequenceClassification.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaForTokenClassification.call
transformers.models.roberta.modeling_tf_roberta.TFRobertaModel.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForCausalLM.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForMaskedLM.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForMultipleChoice.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForQuestionAnswering.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForSequenceClassification.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormForTokenClassification.call
transformers.models.roberta_prelayernorm.modeling_tf_roberta_prelayernorm.TFRobertaPreLayerNormModel.call
transformers.models.segformer.modeling_tf_segformer.TFSegformerForImageClassification.call
transformers.models.segformer.modeling_tf_segformer.TFSegformerForSemanticSegmentation.call
transformers.models.segformer.modeling_tf_segformer.TFSegformerModel.call
transformers.models.vision_text_dual_encoder.modeling_tf_vision_text_dual_encoder.TFVisionTextDualEncoderModel.call
transformers.models.vision_text_dual_encoder.modeling_tf_vision_text_dual_encoder.TFVisionTextDualEncoderModel.from_vision_text_pretrained

docs/source/en/tasks/asr.md
docs/source/en/tasks/audio_classification.md
docs/source/en/tasks/document_question_answering.md
docs/source/en/tasks/idefics.md # cause other test failing
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allow to add comment (despite not very pretty)

@@ -54,7 +54,7 @@ def clean_doctest_list(doctest_file: str, overwrite: bool = False):
all_paths = []
with open(doctest_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
line = line.strip().split(" ")[0]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

take into account the potential comments in some lines of utils/not_doctested.txt

removed_content = set(old_content.split("\n")) - set(new_content.split("\n"))
removed_content = {x.split(" ")[0] for x in old_content.split("\n")} - {
x.split(" ")[0] for x in new_content.split("\n")
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same: deal with the potential comments

src/transformers/models/blip/modeling_tf_blip_text.py
src/transformers/models/blip_2/configuration_blip_2.py
src/transformers/models/blip_2/convert_blip_2_original_to_pytorch.py
src/transformers/models/blip_2/modeling_blip_2.py # cause other test failing
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently, src/transformers/models/blip_2/modeling_blip_2.py and docs/source/en/tasks/idefics.md cause problem to other tests

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Nov 3, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing!

docs/source/en/tasks/asr.md
docs/source/en/tasks/audio_classification.md
docs/source/en/tasks/document_question_answering.md
docs/source/en/tasks/idefics.md # cause other test failing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not able to parse this comment is it saying:

  • Because other tests are failing
    or
  • Causes other tests to fail
    ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these 2 files, if run in doctesting, cause other tests to fail.

I will change it to causes other tests to fail

@ydshieh ydshieh merged commit af8d1dc into main Nov 3, 2023
8 checks passed
@ydshieh ydshieh deleted the fix_doctest_003 branch November 3, 2023 11:47
EduardoPach pushed a commit to EduardoPach/transformers that referenced this pull request Nov 19, 2023
* fix

* update

* update

* fix

---------

Co-authored-by: ydshieh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants